Papers with training complexity
Generative Deduplication For Socia Media Data Selection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Social media data exhibits severe redundancy due to its noisy nature, leading to increased training time and model bias in its processing. |
| Approach: | They propose a new framework for deduplication of social media data by removing semantically duplicate data from the model and add time-dimensional Gaussian noise to reduce training complexity. |
| Outcome: | The proposed framework can reduce training samples while improving performance over baselines. |
Making Pre-trained Language Models Better Learn Few-Shot Spoken Language Understanding in More Practical Scenarios (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing few-shot Spoken Language Understanding models need to be trained on a set of data-rich source domains and adapt to the target domain with a few examples. |
| Approach: | They propose a scenario where only a pre-trained language model and a few labeled examples are used to train few-shot SLU models. |
| Outcome: | The proposed model outperforms existing models on few-shot settings by reducing the number of slot labels and reducing training complexity. |
LightVLP: A Lightweight Vision-Language Pre-training via Gated Interactive Masked AutoEncoders (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing vision-language pre-training models use multi-modal encoders to encode image and text, causing noisy training corpora. |
| Approach: | They propose a vision-language pre-training framework with two autoencoders for efficient training . they propose masked tokens and a gated interaction mechanism to cope with noise . |
| Outcome: | The proposed model achieves 2.2% R@1 gains on COCO Text Retrieval and 1.1% on refCOCO+ on six datasets. |
ProUIE: A Macro-to-Micro Progressive Learning Method for LLM-based Universal Information Extraction (2026.findings-acl)
Copied to clipboard
Wenda Liu, Song Zhigang, Shuai Nie, Guangyao Liu, Lisung Chen, Binyu Yang, Yaran Chen, Peng Zhou, Hongzhen Wang, Yuchen Liu, Wenyue Hu, Jiaming Xu, Runyu Shi, Ying Huang
| Challenge: | ProUIE improves universal information extraction (UIE) without external information . many LLM-based methods rely on extra schema cues, external resources or complex alignment and verification pipelines . |
| Approach: | They propose a Macro-to-Micro progressive learning approach that improves UIE without external information. |
| Outcome: | ProUIE outperforms instruction-tuned baselines on average for NER and RE while using a smaller backbone. |
CLIP-UP: A Simple and Efficient Mixture-of-Experts CLIP Training Recipe with Sparse Upcycling (2025.findings-emnlp)
Copied to clipboard
Xinze Wang, Chen Chen, Yinfei Yang, Hong-You Chen, Bowen Zhang, Aditya Pal, Xiangxin Zhu, Xianzhi Du
| Challenge: | Mixture-of-Experts (MoE) models are crucial for scaling model capacity while controlling inference costs. |
| Approach: | They propose an alternative training strategy that converts a dense CLIP model into a sparse MoE architecture. |
| Outcome: | The proposed training strategy outperforms dense models on COCO and Flickr30k benchmarks. |
TLoRA: Task-aware Low Rank Adaptation of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing low-rank Adaptation (LoRA) methods address only one factor, often at the cost of increased training complexity or reduced practical efficiency. |
| Approach: | They propose a low-rank Adaptation framework that optimizes initialization and resource allocation at the outset of training. |
| Outcome: | The proposed framework performs excellently across various tasks while reducing the number of trainable parameters. |
SiLP: Enhancing Non-Dominant Language Capabilities with a Selective Bidirectional Language Projection Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve performance of large language models rely on additional training objectives or language-specific parameters. |
| Approach: | They propose a bidirectional language projection framework that enables efficient multilingual alignment and language shift using the intrinsic parameters. |
| Outcome: | The proposed framework improves performance of non-dominant languages and improves internal representations. |